Removing Documents

This article provides an overview and how-to instructions for removing documents within Ditto.

See Document Model for more information on documents in Ditto.

This article provides an overview and how-to instructions for removing documents using the EVICT DQL operation.

• Evict Method
• Coordinating Evictions
• Soft-Delete Pattern
• Considerations

Evicting Data

The EVICT method, once invoked, immediately removes the specified document from the local Ditto store, making it inaccessible by local queries.

For complete DQL syntax, see EVICT.

Although the document you evicted is removed from the local Ditto store, the document stored within remote Ditto stores persists.

To prevent the evicted data from reappearing on the screen in a single flicker, make sure to stop subscriptions before you call EVICT; otherwise, the subscription remains active and even if you reset the data in your end-user environment, the evicted data momentarily reappears.

Sample EVICT Query

Swift

await ditto.store.execute("EVICT FROM cars WHERE _id = '123'");

Evicting Multiple Documents in a Collection

The EVICT operation functions based on a condition, allowing updates to multiple documents simultaneously.

For example, the following snippet, once executed, purges all blue cars stored in the local Ditto store.

Swift

let result = await ditto.store.execute(
  "EVICT FROM cars WHERE color = 'blue'");

result.mutatedDocumentIDs.forEach() { print($0) }

Using Evict with Sync Subscriptions

If subscriptions are not properly managed prior to executing evictions, you may inadvertently disrupt the intended state, resulting in inconsistencies and unexpected behavior.

For example, if you have an active subscription for fetching 'blue' cars and you subsequently evict the document with the ID '123456' that matches the replication query, connected peers reinstate it in your local Ditto store. In other words, without modifying the subscription first, peers in the mesh will replicate the evicted document back to the local peer that evicted it.

Therefore we encourage careful management of subscriptions and evictions. To remove documents with active subscriptions, you must first cancel the relevant subscription before calling the EVICT method.

Timing Subscriptions and Evictions

In addition, take a balanced approach when using the Subscribe and Evict methods. Consider the advantages and drawbacks of each method and use them as appropriate for specific needs and requirements.

Key considerations for using Subscription and Eviction methods include:

• Use Subscribe to sync more data across connected peers in the mesh; however, be mindful of potential increased network usage, which may degrade sync performance.
• Use Evict to manage local storage capacity and improve performance by routinely purging data stored locally.

Coordinating Evictions

If you want to indicate that a batch of documents are irrelevant and, although they are to be retained, should not sync across peers, add the isSafeToEvict field to the document property tree. Then, use a method to alert clients to flag any documents they consider irrelevant.

Ditto Document

{ "\_id": "abc123", "color": "red", "mileage": 40000, "isSafeToEvict": true, "createdAt": "2023-05-22T22:24:24.217Z" }

To ensure that peers continue replicating documents that are considered relevant, incorporate isSafeToEvict == false into their sync subscription query.

This approach restricts replication only to documents that peers mark as ‘true’ for isSafeToEvict. Once flagged, the peers clear irrelevant documents from their caches, all the while normal transactional operations continues without interruption.

Swift

await ditto.store.execute("EVICT FROM cars WHERE isSafeToEvict = true");

Referencing Previously Evicted Documents

Once removed, you can reference the evicted document using the mutatedDocumentIDs method on the result.

Soft-Delete Pattern

If you need a data recovery option, instead of permanently removing the data from the local Ditto store like EVICT, opt for a soft-delete pattern.

A soft-delete pattern is a way to flag data as inactive while retaining it for various requirements, such as archival evidence, reference integrity, prevention of potential data loss due to end-user error, and so on.

Adding a Soft-Delete Flag

To add a soft-delete pattern, set the isArchived field value to true:

Ditto Document

{
  "_id": "123",
  "color": "blue",
  "isArchived": true // add this field
}

Querying Non-Archived Documents

To query to monitor documents that are NOT* *archived, establish a live query where isArchived is set to false, and then construct your live query callback.

It’s likely that the isArchived field is set lazily (i.e. has no value until it is true), so you can use the coalesce() function to automatically return false if the value is unset. The following code demonstrates searching for documents that are unarchived:

Swift

let result = await ditto.store.execute("""
  SELECT *
  FROM cars
  WHERE coalesce(isArchived, false) = false
  """)

Removing Soft-Delete Flag

To remove the flag and reactivate the document, set the isArchived field to false:

Swift

await ditto.store.execute("""
  UPDATE cars
  SET isArchived = false
  WHERE _id = '123'
  """)

Removing Data from Big Peer

EVICT is not currently available for the Big Peer, although it’s in development. To learn more about removing data from your Big Peer instance, see Writing: HTTP (Legacy).

Considerations

To mitigate the risk of memory leaks, performance degradation, crashes, data loss, and, if applicable, reduced battery life, it is critical that you implement a thoughtful memory management strategy in your app.

Depending on your use case, use either the Evict method or apply the soft-delete pattern to implement tools to help optimize memory usage in your app.

When planning your approach to memory management in your app, use the following criteria to help you during the decision-making process:

Consideration	Recommendation
Access frequency and relevance	Ensure memory is allocated only to the most relevant and frequently accessed documents by establishing an automatic process that evicts documents that are: - Accessed less frequently - No longer relevant or needed
Time-based data	Establish an automatic process to evict or remove time-based data older than a minimum of seven days. (Until expired, time-based data remains accessible by way of local queries.)
Permanent data loss	If documents are evicted from a local peer and don’t exist on any other peer this data is lost and is unrecoverable.

Access Frequency and Relevance Considerations

In peer-to-peer system design, there are technical tradeoffs between the amount of data synced across peers and the timeliness of access to synced data:

• The greater the amount of data synced, the more timely offline read access becomes. That is, database resilience in offline scenarios increases when more documents are being synced across distributed peers.
• The fewer the number of documents replicated, the less the likelihood that peer devices run out of disk space and experience memory leaks, and the performance of the peer-to-peer mesh network that interconnects them degrades.

For considerations on using the Evict and Subscribe methods, see Timing Subscriptions and Evictions.